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In this article, we determine how to facilitate the analysis of serious games 
so that teachers could effectively integrate them in their teaching. The aim is 
to identify the mechanisms that would make serious games exploitation 
useful. We propose a method for the analysis of serious games that is based 
on the separation of their components along three phases. In addition, we set 


up a platform based on a data analysis process that is composed of six steps 
that help to set the basis of a verification procedure that targets the content 
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1. INTRODUCTION 

Currently, digital technology has taken over most areas of our lives, including in education. At a 
time when the time is reserved by users for electronic media has increased which generally has made video 
games a favorite hobby. The serious game (SG) is a new tool to learn in a ludic way and is increasingly 
present in the world of teaching. They cover almost all areas [1]-[3]: sciences, economics, politics, 
management, health, and ecology. Today, the new generation of learners (NG) has very different 
expectations, demands, and aspirations. The NG is adapted to the culture of video games, since childhood, 
and one of its salient characteristics is the heavy use of technical/technological devices. Therefore, it is not 
easy for this NG to adapt to the traditional pedagogical methods on which teaching is generally based on. To 
achieve the intended educational results and create a favorable climate, teachers must motivate their learners 
and provoke interaction. Researchers [4], [5] claim that a teacher's educational choices can be decisive in 
how students engage in a course. Likewise, Sauvé et al. [6] confirm that educational activities should be 
diversified to give students the opportunity to make choices and break out of the routinely maintained 
methods and approaches. According to Hmelo-Silver and Barrows [7], teaching is a complex task and must 
use a set of strategies to make learning effective. Indeed, Douglas et al. [8] show that teachers should focus 
on improving their teaching methods by adopting a variety of teaching methodologies, technics, and material. 
Moreover, according to the researchers [9], it is essential that the teaching style must be effective enough to 
be well understood, and that learning, as an activity, can be conducted with learners. Besides, the 
environment in which learning takes place must adapt to the needs and expectations of learners. Game-based 
pedagogy could be a solid mechanism for learning, especially during times of crises and pandemics. Lately, 
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and still, we have noticed how that many digital tools are important to our life in general, and to education in 
specific. Several studies [10], [11] confirm the utility of SGs in the field of education. Indeed, the studies of 
[12]-[14] show their contributions in terms of motivation and attractiveness for learners. 

One of the main facilitators of game adoption in the learning process is the teacher. Therefore, this 
section explains, through some studies, the role these controlling agents are ready or not, to SGs as a 
teaching/learning strategy. Hence, several studies [10], [11], [15], evaluate the effectiveness of the use of SGs 
in the results of the students. To our knowledge, few studies have been interested in the perception of the 
teachers, especially in regard to the adoption of such opportunities in their normal teaching practice. Indeed, 
if teachers are not motivated, or if they lack a positive attitude towards the implementation of SGs in a 
teaching-learning sequence, it will be difficult for them to adopt a game-based pedagogy. Thus, the studies of 
[16], [17], have tried to show to what extent the role of teachers is crucial in the creation of learning 
environments based on the game. Beddaou et al. [18] confirm that the relevance of an SG depends on the 
nature of the game itself, but also on how it is used in the teaching-learning process. Becker [19] too asserts 
that a game must necessarily be tested by the teacher before been the subject of any pedagogical integration. 
In addition, Kangas et al. [20] points out that adopting SGs requires the involvement of teachers before, 
during, and after any game-based intervention. Studies from [21], [22] state that teachers do not easily 
integrate SGs into their actual teaching practices. Indeed, according to Hébert et al. [23], assert that there is a 
need for experimental research that addresses the challenges of implementing game-based pedagogy in 
various classroom contexts, taking into account the obstacles which teachers themselves are confronted to 
optimize this methodology. 

The process of adopting a game-based pedagogy requires, unconditionally, accompanying teachers 
with tools that facilitate understanding and effective implementation during a learning activity. The 
classification of Djaouti et al. [24], in fact, has helped to classify serious games on a website entitled: 
Scientific Laboratory for Research on Video Games (Ludoscience). However, the scope of this classification 
remains limited for teaching needs due to the lack of any concrete correspondence between the elements of 
the grid and the needs expressed by the teaching staff. A simplified analysis grid, resulting from the research 
work of Sanchez [25], was presented on the website The Digital Games Bank of the Academy of 
Montpellier. But the latter focuses on commitment, autonomy, social interactions, freedom, rules and 
rewards, mistakes, failures, and other emotional aspects, as well as the content of the game. Yet, Mathieu 
[26] proposed an analysis grid that targets serious games for science and technology teaching based on the 
research work that Becker proposed [19]. The study that is conducted by Emin-Martinez [27] proposed an 
adoption model based on Rogers’ diffusion of innovation theory [28]. According to Surry and Farquhar [29], 
this theory helps to better understand and facilitate the adoption of new educational products. Rogers’ theory 
has been widely applied to various educational technologies [30] including SGs with the study of Kebritchi 
[22]. 

The purpose of this research work, on one hand, is to propose a method of analysis to evaluate the 
ludic and pedagogical potential of SGs. On the other hand, to establish an environment facilitating the 
understanding and effective integration of SGs in the teaching process. After this introduction, the second 
section is reserved for the presentation of the research method, which presents the proposed method of 
analyzing SGs and the used experimental protocol to evaluate this method. Then, the third section is devoted 
to the analysis of the obtained results from the experiment and the discussion of the results. Finally, the 
conclusion is presented in the fourth section. 


2. RESEARCH METHOD 
2.1. Pedagogical act in the case of a serious game 

Recent research on the integration of SGs into the teaching process has undergone significant 
development. Many researchers have shown that the integration of SGs in education has a positive impact on 
the learning of learners. Silseth's study [31] states that SGs are valuable tools for facilitating dialogue in the 
classroom. However, many factors prevent the effective integration of these games in the educational 
environment. Indeed, the study by Arnseth etal. [32] asserts that there are still few empirical studies 
exploring how games are used in education. It is necessary, according to Molin [33], to examine how 
teachers position themselves concerning games. In general, the proper functioning of an SG during a game- 
based learning situation is conditioned by the establishment of coherent and clear communication between 
the teacher, the learner, and the game to obtain the best ratio between the pleasure of the game and the 
teaching effectiveness. We have defined seven relationships in our study of the analysis of the pedagogical 
act in the case of a serious game. Figure 1 presents a diagram of the pedagogical act in the case of a serious 
game. 
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Thanks to this study, it can be ensured that the role of the teacher is to choose the SG, which is best 
suited to his learners, his teaching objectives, his method of work, and also to help the learner achieve the 
intended objectives. Table 1 presents the concept of the links between the teacher, the learner, and the game. 
Thanks to this analysis study, which targeted the position of the game concerning the teacher and the learner, 
we have defined two verification processes for the content of the game used by the teacher to judge the 
relevance of the game for a specific learning situation. Figure 2 shows a description of the steps in the SG 
verification process. The effective integration of SG into the teaching process requires a double check. On the 
one hand, the content of the game must be compatible with the knowledge of the learners and on the other 
hand, the content of the game must meet the pedagogical needs of the teacher. 
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Figure 1. Diagram of the pedagogical act in the case of a serious game 
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Table 1. Pedagogical relationships in the learning situation through serious game 
Relationship Role 
PLAY The learner acquires knowledge through a game-based learning situation. The learner 
can be in contact with the sources of knowledge, to build their knowledge as 
effectively as possible. 


USE The teacher must have a good knowledge of the used games in order to know what 
disciplinary knowledge and transversal skills are worked in these games. 
ORIENT The teacher would observe the learner and systematically correct or assist the learner. 
ADAPT The teacher chooses the game and then adapts it according to the content and the 
targeted skills. Moreover, it organizes space and time (place, material, duration...). 
MODELLING The educational content can be modelled as a meaningful and intelligible object. This 
modelling constitutes pedagogical structuring in a ludic context. 
IMPLEMENTATION This is step reserved for the design of the elements constituting the game. 
MOTIVATE The game promotes motivation in the face of a given subject. Indeed, it creates a 


positive pedagogical climate in which the error is de-dramatized and the center of 
attention is moved from the pedagogical content to the ludic task to be performed. 


USE GAME 


MOTIVATE 


IMPLEMENTATION 
MODELING 


Figure 2. Steps in the verification process 
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2.2. Serious game analysis steps 

Although several studies confirm the positive impact of SGs by basing on learners' learning 
outcomes, there is still a lack of appropriate measures and methods to assess the different quality 
characteristics in SGs. Also, the level of expertise of the designers and developers of SGs influences their 
qualities. In addition, only the teacher can choose adapted games to the prerequisites of his learners, his 
pedagogical objectives, and his work method. Since some knowledge may be transmitted more effectively 
through a lecture course, others will need an SG to allow learners to acquire it. Therefore, it will be difficult 
to judge the qualities and effectiveness of an SG without methods of evaluation of the different ludic and 
educational components of the game. 

Calderón et al. [34] summarize the characteristics that have been used in the literature to evaluate 
serious games in 18 characteristics. The 18 characteristics are game design, user satisfaction, usability, 
usefulness, understandability, motivation, performance, playability, pedagogical aspects, learning results, 
engagement, user experience, efficiency, social impact, cognitive behavior, pleasure, acceptance, and user 
interface. The study of [35] shows that most approaches to how to evaluate educational games do not provide 
an explicit definition of the purpose, measures or, data collection instruments. 

Given the lack of methods that facilitate the analysis of SGs, we have proposed a method of analysis 
in three phases, called SIU (structure, interface, and use) that allows to ensure the balance between ludic 
elements (motivation and commitment) and educational content (effectiveness of learning). Figure 3 shows a 
description of the steps in the proposed analytical method: 

a. Structure phase: ensures the level of compatibility between the pedagogical aspect and the ludic aspect of 
an SG. This phase allows giving results on the level of compatibility of the educational objectives to be 
achieved with the ludic scenario of the game. 

b. Interface phase: confirms the level of compatibility of the utility and usability of an SG with the 
educational objectives and characteristics of the learners. This phase permits giving results on the quality 
of the technical elements of the game environment and the quality of the interactions provided to the 
learner to achieve defined objectives. 

c. Use phase: ensures the level of compatibility of the understanding conditions and the degree of 
knowledge acquisition of an SG with the characteristics and knowledge of the learners. This phase 
provides results on the understanding and control conditions of the learners (total score, number of 
mistakes, time spent on each task, and others) and the degree to which knowledge and skills are acquired. 
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Figure 3. Steps in the analysis of a serious game 


The SIU method is based on a hierarchical representation at several levels representing all the 
elements constituting the SG. Figure 4 shows an example of a hierarchical representation for the intrinsic 
motivation factors. The SIU method is based on a step-by-step control and decision-making process to analyze 
the SG effectively. Figure 5 presents an extract of the evaluation of the validity of the content of the SG. 

The SIU method makes it possible to determine the evaluation criteria to measure the ludic and 
pedagogical dimension of an SG to judge the relevance of the game for a specific learning situation. Table 2 
presents the criteria for assessing the validity of the SG content. The SIU method makes it possible to exploit 
the potential of SGs in education without involving teachers in overly complex procedures that require 
computer skills and also to help teachers to understand and effectively implement SGs during a learning 
activity. 
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Figure 4. Example of hierarchical representation for intrinsic motivation factors 
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Figure 5. Extract from analysis of the validity of the content of a serious game 


Table 2. Criteria for evaluating the validity of serious game content 
TASK - Educational content 


Setting Criteria Meaning 
Setting | Relevance Relates exactly to what is requested 
Setting 2 Consistency Logical links between the elements, everything is organized 
Setting 3 Precision Detailed, clear, concise 


2.3. Evaluation of the proposed method 

In this part, we present the methodology followed to evaluate our approach. This study was carried 
out in two stages during a period of 9 months, going from May 2020 to January 2021. The first six-month 
stage was devoted to the implementation of a platform based on our approach, called serious game analysis 
(SGA). Then, in the second three-month phase, we have established an evaluation protocol that measures the 
efficiency and usefulness of the approach we are proposing. 


2.3.1. Presentation of the SGA platform 
We have set up a digital environment that allows teachers to visualize, analyze and evaluate SGs ina 


six-step process: i) discover the game; ii) verify pedagogical compatibility; iii) verify motivational aspects, 
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iv) study the rules, constraints, and the penalty/rewards system; v) check the visual and auditory elements; 
and vi) study the conditions and criteria for the assessment of prior learning. Moreover, SGA users can 
perform three types of actions: i) perform an analysis of a game to generate an analysis report, ii) compare 
analysis published results by other teachers, and iii) import published analysis results to generate a 
personalized report. Each game has associated a list of details such as classification, number of published 
reports, results of analysis, and more. The teacher can then use these details to assess the quality and 
usefulness of the game against predefined teaching areas. 


2.3.2. Experimental method 

The purpose of this section is to explain the experimental protocol used to evaluate the effectiveness 
of the SIU method. We have realized two experiments. The objective of the first experiment is to study the 
usefulness and effectiveness of the SGA tool. The second experiment aims to study the appreciation of 
teachers and students regarding the precise evaluation criteria on the SGA tool analysis report in a real context. 

a. First experiment 

To validate the effectiveness of the proposed method, we performed an experimental study in five 
phases: i) use of the platform in a real context, ii) collect feedback from teachers on the analysis process SGs, 
iii) conducting a questionnaire with teachers, seeking to verify the level of perceived efficiency of the 
platform, iv) analyzing the collected data by questionnaire with a statistics and data management software 
package, and v) analysis of the results of the last two phases. 

We tested our approach in using the SGA platform. For this, a sample of 40 teachers who have 
agreed to participate in such an experiment, 24 teachers from the Polydisciplinary Faculty of Taza and 
16 teachers from the Polidisciplinary Faculty of Nador, in the following disciplines: Computer Sciences, 
Physics, Economics, Biology, English and French. We sent out a video and a user guide to the teachers that 
explained the description of the SIU method and how to use the SGA platform. 

Only four of the participants are experienced users of the games and have been using them for two 
years. The rest are using the game for the first time. The disciplines that are interested in this study are 
characterized by a level of teachers’ skills that are sufficient concerning the use of computer software. 

The duration of the experiment is variable for each teacher, as it depends on the duration of the 
tested game and also on the level of the teachers' experience concerning the game. Some teachers played the 
game twice to familiarize themselves with the game (content, controls, how to access levels, and others). To 
better analyze these results, we also carried out an evaluation based on a questionnaire to assess the teachers’ 
point of view regarding the possible use of SGs and to collect their opinions on the qualities of the platform 
such as the quality of the games offered, utility, interface qualities, and user satisfaction. We used a 5-point 
likert scale from 1 (strongly disagree) to 5 (strongly agree) to determine user responses. 

b. Second experiment 

To support teachers in the use of SGs in the classroom, we organized an experiment of two working 
sessions with four teachers in two different establishments (Polydisciplinary Faculty of Taza and 
Polydisciplinary Faculty of Nador). The first session was devoted to the choice of the SG by the teacher 
using the SGA platform. Table 3 shows the games analyzed by the teachers. 

In the second session, we prepared and conducted the game session with the teachers based on the 
SGA report. The session consists of the game phase and the debriefing phase. The objective was to help 
teachers adopt SGs in their educational contexts and to observe the impact of SG on students' motivation and 
behavior. 


Table 3. Presentation of the games of the experimental study 


Game Realization Ludic video type Thematic and target audience 
CodeCombat = CodeCombat Inc. Game of programming Game teaches different programming languages (JavaScript, 
Python ...) as well as the fundamentals of computer science. 
Leuco’ war Philippe Game of immunology Game allows students to visualize the functioning of the immune 
COSENTINO system following an infection. 
Cartel euros Creatiel LLC Game of management Game allows students to learn management. 
3000 


3. RESULTS AND DISCUSSION 

Of the 40 teachers, more than 90% did not use SGs in their teaching. However, during the 
experiment, 65% of teachers played several times the chosen game. It is to familiarize themselves with the 
content and operational details. 
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3.1. Results 

In our experiment, the SGs evaluated had diverse themes, most of which are of the type shooting 
game, simulation game, strategy game, and puzzle game. Player-game interactions were most often done 
through drop-down menus. Regarding the scoring methods, some games had a score per stage, but most had 
an overall score. 


3.1.1. First experiment 

During experimentation with the use of the SGA platform, 72.5% of teachers generate a summary 
report of the SG chosen. Only 10% of participants stopped the analysis in the first phase by clicking on 
“Content not compatible with educational objectives”. However, 7.5% of teachers have identified difficulties 
in understanding the rules of the game and 10% of users indicated difficulties in setting assessment 
conditions. We also noted that 2.5% of participants analyzed two games to familiarize themselves with the 
learning management system (LMS) platform. Some games have been analyzed several times by different 
profiles. 

To better analyze these results, we also performed an assessment based on a questionnaire 
comprising 10 questions to assess the qualities of SGA. We have added a question to check the use of SGs. 
Figure 6 shows the mean and standard deviation of the results of the questionnaire concerning the qualities of 
the platform such as the qualities of the games offered, the usefulness, the interface qualities, and the user 
satisfaction. Among the 40 teachers, 82.5% of teachers said the platform helped them to change their 
appreciation of the need to use games during a learning situation. 10% are neutral on this issue, and 7.5% 
disagree with it. 


Averages of the questionnaire results 


3.5 m Question 1 


3 Question 2 


Average results 


Quality Utility Interface Usability Satisfaction 


Questions 
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Quality Utility Interface Usability Satisfaction 
Questions 


Figure 6. Mean and standard deviation of the results of the questionnaire concerning the qualities of SGA 
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3.1.2. Second experiment 

We conducted a second experiment by following two avenues to assess the effectiveness of the 
proposed method. Firstly, we evaluated the effectiveness of SGs in teaching by analyzing the influence of 
SGs on student motivation and teacher appreciation. Secondly, we assessed the effectiveness of the SGA 
platform by analyzing the usefulness of the teacher-generated analysis report. 

Of the 120 students that participated in the survey, 95% of students did not have a negative 
opinion about the value of using an SG in a learning session, and 92.3% of them found that using a game 
facilitates some learning. The students generally appreciated the initiative and its applicant of this type of 
pedagogy. The teachers agreed on the need for games during a learning situation and also the need for a prior 
assessment of game content. The four teachers found that the precise evaluation criteria on the analysis report 
generated via SGA make it possible to determine whether the objectives of the game are compatible with the 
intended pedagogical objectives, and therefore whether the game is effective in helping the learners. Teachers 
also indicated the need to add other phases to facilitate the process of adopting SGs in education. 


3.2. Discussion 

Serious games could be an effective strategy to improve the teaching process. However, it is 
necessary to assess beforehand the conformity of their contents with a specific learning situation. 75% of 
teachers found that the specific assessment criteria on the SGA analysis report help to determine whether the 
game content is effective in supporting learning. Only 25% of teachers had difficulty for several reasons, 
which hampered their involvement and thus their results. In addition, 20% of users used the comment feature 
to explain the decision to stop the analysis, allowing the system to classify games according to new criteria. 
Results from the questionnaire showed that 82.5% of teachers were overall satisfied with the use of SGA and 
agreed on the need to use SGs to improve teaching. During the experiments, teachers mentioned the need for 
vocational training related to game-based pedagogy. 

Emin-Martinez [27] used an adoption model based on Rogers’ theory [28] (the five phases and the 
five attributes) as a reading grid following an experience of accompanying teachers in the adoption of a 
game-based pedagogy. Thus, Mathieu [26] proposed an analysis grid of SGs for the teaching of science and 
technology-based on Becker’s research work [19]. In our study, we implemented an SGA platform based on 
the SIU method that facilitates the understanding and effective use of SGs in education without requiring 
teachers to adopt new roles that are too complex and require computer skills. In general, the SIU method is 
characterized by three main dimensions; i) a hierarchical representation at several levels, ii) a process of 
control and decision-making by stages, and iii) a process of determining the evaluation criteria to assess the 
ludic and educational dimension of an SG. Generally, the obtained results from the experiments show that the 
SIU method helps and accompanies teachers in the decision-making concerning the adoption of game-based 
pedagogy. To make this work to be all the more relevant, some points could be improved, such as the 
addition of a phase to the SIU method explains the protocol for the concrete use of SGs in education 
according to specific cases. 


4. CONCLUSION 

Currently, a serious game is an essential tool for learning and teaching. However, the research has 
neglected the role of the teacher in selecting and evaluating this tool for effective teaching. Also, the process 
of adopting a game-based pedagogy requires the accompaniment of teachers with tools facilitating the 
understanding and effective implementation of this strategy in education. In this article, we proposed a 
method for the analysis of serious games based on the separation of their components according to three 
phases, namely, the phase of the structure, the phase of the interface, and the phase of use. This method is 
characterized by three main dimensions: i) a hierarchical representation at several levels, ii) a process of 
control and decision-making by stages, and iii) a process of determining the evaluation criteria to assess the 
ludic and educational dimension of a serious game. Next, we developed the SGA tool based on the SIU 
method to facilitate the understanding and effective use of serious games in education without teachers 
having to adopt new roles that are too complex and require computer skills. 

Overall, the SGA tool assessment protocol has yielded positive results relative to the usefulness and 
effectiveness of the analysis reports. We obtained over 75% satisfaction with the specific evaluation criteria 
for the SGA analysis report. In addition, 82.5% of teachers expressed that the use of the platform helps them 
to change their view on the need to use serious games in education. Finally, the lack of professional training 
for teachers on strategies for game-based pedagogy presents the first barrier to the use of the game in 
teaching. We believe that the addition of a phase to the SIU method could improve the accuracy of the 
selection of serious games for effective learning and also improve the process of adoption of serious games 
by teachers. 
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